Search for: All records

Robust Markov Decision Processes (MDPs) offer a promising framework for computing reliable policies under model uncertainty. While policy gradient methods have gained increasing popularity in robust discounted MDPs, their application to the average-reward criterion remains largely unexplored. This paper proposes a Robust Projected Policy Gradient (RP2G), the first generic policy gradient method for robust average-reward MDPs (RAMDPs) that is applicable beyond the typical rectangularity assumption on transition ambiguity. In contrast to existing robust policy gradient algorithms, RP2G incorporates an adaptive decreasing tolerance mechanism for efficient policy updates at each iteration. We also present a comprehensive convergence analysis of RP2G for solving ergodic tabular RAMDPs. Furthermore, we establish the first study of the inner worst-case transition evaluation problem in RAMDPs, proposing two gradient-based algorithms tailored for rectangular and general ambiguity sets, each with provable convergence guarantees. Numerical experiments confirm the global convergence of our new algorithm and demonstrate its superior performance. 

ABSTRACT This integrative literature review analyzes the corpus of biology education research published in the main biology education journals of major professional societies. The goal of this analysis is to determine which approaches (including groups of focus, research methods, and settings/perspectives) from social science fields (i.e., psychology, sociology, and anthropology) are utilized in published peer-reviewed biology education research relating to diversity, equity, and inclusion (DEI). Scoping how social science approaches are used in this area is important to understanding whether biology education research could benefit from complementary approaches that might advance praxis. This analysis found that research informing the biology education community draws heavily from psychological perspectives that are overwhelmingly not disaggregated (78% of articles identifying a group lumped the participant together), are by far more quantitative (58% used survey, 26% grades, 20% school data) than qualitative (17% used interview, 10% observation), and did not adopt structural approaches (72%). The addition of missing contributions from social science is critical to advancing interventions to broaden STEM participation, given that merging paradigms can offer more robust, multi-level explanations for observed phenomena. This has important implications for education, biology education, biology education research, social science, and research in related STEM fields.